Rule Value Reinforcement Learning for Cognitive Agents

RVRL (Rule Value Reinforcement Learning) is a new algorithm which extends an existing learning framework that models the environment of a situated agent using a probabilistic rule representation. The algorithm attaches values to learned rules by adapting reinforcement learning. Structure captured by the rules is used to form a policy. The resulting rule values represent the utility of taking an action if the rule`s conditions are present in the agent`s current percept. Advantages of the new framework are demonstrated, through examples in a predator-prey environment

From Agent Game Protocols to Implementable Roles

kostas.stathis-at-cs.rhul.ac.uk Abstract. We present a formal framework for decomposing agent interaction protocols to the roles their participants should play. The framework allows an Authority Agent that knows a protocol to compute the protocol’s roles so that it can allocate them to interested parties. We show how the Authority Agent can use the role descriptions to identify problems with the protocol and repair it on the fly, to ensure that participants will be able to implement their role requirements without compromising the protocol’s interactions. Our representation of agent interaction protocols is a game-based one and the decomposition of a game protocol into its constituent roles is based upon the branching bisimulation equivalence reduction of the game. The work extends our previous work on using games to admit agents in an artificial society by checking their competence according to the society rules. The applicability of the overall approach is illustrated by showing how to decompose the NetBill protocol into its roles. We also show how to automatically repair the interactions of a protocol that cannot be implemented in its original form.

Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

The KGP model of Agency for Decision Making in e-Negotiation

We investigate the suitabilility of the KGP (Knowledge, Goals, Plan) model of agency for autonomous decision making in dynamically changing environments. In particular, we illustrate how this model supports the decision making process of an agent at different levels, while the agents generates goals, plans for these goals, and selects actions to achieve the goals that it has planned for. We also exemplify the approach by illustrating how the model and a prototype implementation in the PROSOCS platform can be adopted to support e-negotiation, using a particular kind of internet auctions as a case study

Gate Stack Dielectric Degradation of Rare-Earth Oxides Grown on High Mobility Ge Substrates

We report on the dielectric degradation of Rare-Earth Oxides (REOs), when used as interfacial buffer layers together with HfO2 high-k films (REOs/HfO2) on high mobility Ge substrates. Metal-Oxide-Semiconductor (MOS) devices with these stacks,show dissimilar charge trapping phenomena under varying levels of Constant- Voltage-Stress (CVS) conditions, which also influences the measured densities of the interface (Nit) and border (NBT) traps. In the present study we also report on C-Vg hysteresis curves related to Nit and NBT. We also propose a new model based on Maxwell-Wagner instabilities mechanism that explains the dielectric degradations (current decay transient behavior) of the gate stack devices grown on high mobility substrates under CVS bias from low to higher fields, and which is unlike to those used for other MOS devices. Finally, the time dependent degradation of the corresponding devices revealed an initial current decay due to relaxation, followed by charge trapping and generation of stress-induced leakage which eventually lead to hard breakdown after long CVS stressing.Comment: 19pages (double space), 7 figures, original research article, Submitted to JAP (AIP

The Geometry of D=11 Killing Spinors

We propose a way to classify all supersymmetric configurations of D=11 supergravity using the G-structures defined by the Killing spinors. We show that the most general bosonic geometries admitting a Killing spinor have at least a local SU(5) or an (Spin(7)\ltimes R^8)x R structure, depending on whether the Killing vector constructed from the Killing spinor is timelike or null, respectively. In the former case we determine what kind of local SU(5) structure is present and show that almost all of the form of the geometry is determined by the structure. We also deduce what further conditions must be imposed in order that the equations of motion are satisfied. We illustrate the formalism with some known solutions and also present some new solutions including a rotating generalisation of the resolved membrane solutions and generalisations of the recently constructed D=11 Godel solution.Comment: 36 pages. Typos corrected and discussion on G-structures improved. Final version to appear in JHE

Sharp Trace Hardy-Sobolev-Maz'ya Inequalities and the Fractional Laplacian

In this work we establish trace Hardy and trace Hardy-Sobolev-Maz'ya inequalities with best Hardy constants, for domains satisfying suitable geometric assumptions such as mean convexity or convexity. We then use them to produce fractional Hardy-Sobolev-Maz'ya inequalities with best Hardy constants for various fractional Laplacians. In the case where the domain is the half space our results cover the full range of the exponent $s \in (0,1)$ of the fractional Laplacians. We answer in particular an open problem raised by Frank and Seiringer \cite{FS}.Comment: 42 page